Digital watermarking in data representing color channels

ABSTRACT

The present disclosure relates to digital watermarking. One claim recites a method to detect two or more different digital watermarks in media. The method includes: receiving captured imagery of the media, the captured imagery comprising a plurality of image frames; for a first image frame applying a first watermark detector to search for a first digital watermark hidden within the first image frame, in which an electronic processor is programmed as the first watermark detector; and for a second image frame applying a second, different watermark detector to search for a second, different watermark hidden within the second image frame, in which an electronic processor is programmed as the second watermark detector. Other claims and combinations are provided too.

RELATED APPLICATION DATA

The present application is a continuation of U.S. patent applicationSer. No. 11/153,901, filed Jun. 14, 2005 (U.S. Pat. No. 7,738,673),which is a continuation in part of U.S. patent application Ser. No.10/818,938, filed Apr. 5, 2004 (U.S. Pat. No. 6,996,252). The 10/818,938application is a continuation of U.S. patent application Ser. No.09/945,243 filed Aug. 31, 2001 (U.S. Pat. No. 6,718,046), which is acontinuation in part of U.S. patent application Ser. No. 09/933,863,filed Aug. 20, 2001 (U.S. Pat. No. 6,763,123), which is a continuationin part of U.S. patent application Ser. No. 09/898,901, filed Jul. 2,2001 (U.S. Pat. No. 6,721,440), which is a continuation in part of U.S.patent application Ser. No. 09/553,084, filed Apr. 19, 2000 (U.S. Pat.No. 6,590,996). Application Ser. No. 10/818,938 is also a continuationin part of U.S. patent application Ser. No. 10/115,582, filed Apr. 2,2002 (U.S. Pat. No. 6,912,295). The 10/115,582 application is acontinuation in part of U.S. patent application Ser. No. 09/945,243,filed Aug. 31, 2001 (U.S. Pat. No. 6,718,046). The 09/945,243application is a continuation in part of U.S. patent application Ser.No. 09/933,863, filed Aug. 20, 2001 (U.S. Pat. No. 6,763,123). The09/933,863 application is a continuation in part of U.S. patentapplication Ser. No. 09/898,901, filed Jul. 2, 2001 (U.S. Pat. No.6,721,440), which is a continuation in part of U.S. patent applicationSer. No. 09/553,084, filed Apr. 19, 2000 (U.S. Pat. No. 6,590,996).Application Ser. No. 10/818,938 is also a continuation in part of U.S.patent application Ser. No. 10/823,514, filed Apr. 12, 2004 (U.S. Pat.No. 7,027,614). The 10/823,514 application is a continuation of U.S.patent application Ser. No. 09/898,901, filed Jul. 2, 2001 (U.S. Pat.No. 6,721,440). The 09/898,901 application is a continuation-in-part ofU.S. patent application Ser. No. 09/553,084, filed Apr. 19, 2000 (U.S.Pat. No. 6,590,996). This application is also related to U.S. Pat. Nos.6,891,959, 6,804,377, 6,614,914, 5,862,260, 5,822,436, 5,832,119 and5,748,783; published U.S. Patent Application No. US 2002-0170966 A1,U.S. patent application Ser. No. 09/186,962, filed Nov. 5, 1998, andInternational Application No. PCT/US96/06618, filed May 7, 1996(published as WO 96/36163). Each of the above U.S. patent documents ishereby incorporated by reference.

FIELD

The present disclosure relates to hiding data in color channels.

BACKGROUND AND SUMMARY

The above mentioned parent applications disclose various techniques forembedding and detecting of hidden digital watermarks.

Digital watermarking technology, a form of steganography, encompasses agreat variety of techniques by which plural bits of digital data arehidden in some other object, preferably without leaving human-apparentevidence of alteration.

Digital watermarking may be used to modify media content to embed amachine-readable code into the media content. The media may be modifiedsuch that the embedded code is imperceptible or nearly imperceptible tothe user, yet may be detected through an automated detection process.

Digital watermarking systems typically have two primary components: anembedding component that embeds the watermark in the media content, anda reading component that detects and reads the embedded watermark. Theembedding component embeds a watermark pattern by altering data samplesof the media content. The reading component analyzes content to detectwhether a watermark pattern is present. In applications where thewatermark encodes information, the reading component extracts thisinformation from the detected watermark. Assignee's U.S. patentapplication Ser. No. 09/503,881, filed Feb. 14, 2000 (now U.S. Pat. No.6,614,914), discloses various encoding and decoding techniques. U.S.Pat. Nos. 5,862,260 and 6,122,403 disclose still others. Each of theseU.S. patent documents is herein incorporated by reference.

Now consider our out-of-phase digital watermarking techniques withreference to FIGS. 1 a and 1 b. In FIG. 1 a, the dash/dot C, M, Y and Klines represent, respectively, cyan, magenta, yellow and black colorchannels for a line (or other area) of a media signal (e.g., a picture,image, media signal, document, etc.). The FIG. 1 a lines represent abase level or a particular color (or gray-scale) level (or intensity).Of course, it is expected that the color (or gray-scale) level will varyover the media signal. FIG. 1 b illustrates the media of FIG. 1 a, whichhas been embedded with an out-of-phase digital watermark signal. Thewatermark signal is preferably applied to each of the color componentdimensions C, M and Y.

In FIGS. 1 a and 1 b, the M and Y channels are represented by onesignal, since these color components can be approximately equal, butseparate signals. Of course, it is not necessary for these components tobe equal, and in many cases the yellow and magenta components are notequal. The illustrated “bumps” (or “tweaks”) in FIG. 1 b represent thedigital watermark signal, e.g., upward and downward signal adjustmentsin relation to a respective color channel at given points over the mediasignal. The tweaks are preferably applied at the same level (or signalstrength). Alternatively, the bumps are applied with a different signalstrength (or tweak level) when compared to one another. Of course, thesetweaks can be embedded over a color channel in a predetermined pattern,a pseudo random fashion, a random fashion, etc., to facilitate embeddingof a digital watermark signal.

For the K dimension (or channel), the digital watermark signal ispreferably embedded to be out-of-phase with respect to the CMY channels.Most preferably, the K channel is approximately 180 degrees out-of-phase(e.g., inverted) with the watermark signals in the CMY color channels,as shown in FIG. 1 b. For example, if a digital watermark signalmodifies each of the color channels at a media' first location with atweak level of say 7, then a tweak level of −7 correspondingly modifiesthe K channel at the media's first location. This digital watermarktechnique is referred to as our out-of-phase (or “K-phase”) digitalwatermark. (We note that if a watermark signal is determined in terms ofluminance, we can assign or weight corresponding tweak levels to therespective color plane pixel values to achieve the luminance valuetweak. Indeed, a tweak can be spread over the CMY channels to achieve acollective luminance at a given media location. The luminanceattributable to the CMY tweak is preferably cancelled or offset by theluminance effect attributable to a corresponding inverted K channeltweak at the give media location. Similarly, if a watermark signal isdetermined in terms of chrominance, we can assign or weightcorresponding tweak levels to the respective color plane pixel values toachieve the chrominance value tweak. Indeed, a tweak can be spread overthe CMY channels to achieve a steady luminance at a given medialocation. The luminance attributable to the CMY chrominance tweaks arepreferably cancelled or offset by the luminance effect attributable to acorresponding inverted K channel tweak at the give media location. Ormore generally, the luminance in a given localized area is preferablysteady or minimal since chrominance tweaks in a first color channelreduces luminance attributable to a chrominance tweaks in a seconddifferent color channel).

Our inventive watermarking scheme greatly reduces watermarkperceptibility. Since the watermark signal for the K channel is appliedapproximately 180 degrees out-of-phase, when compared to the respectivetweaks applied to the C, M and/or Y channels, the watermark visibilityis greatly reduced. The visibility reduction is achieved by theeffective cancellation of perceived luminance changes when the CMYKimage is viewed or printed. Indeed, combining an inverted watermarksignal “tweak” or “bump” in a K channel with a correspondingnon-inverted watermark signal tweak or bump in the CMY channelseffectively cancels an overall perceived luminance change for a givenarea (e.g., a pixel or block of pixels)—greatly reducing visibility ofthe digital watermark.

The present disclosure discloses a new data hiding technique based onour out-of-phase technology. According to one implementation of thepresent disclosure, an image is hidden in or carried by a media signal.The hiding is accomplished with our out-of-phase embedding techniques.The image can be a photograph, a graphic, a barcode (1-D or 2-D), etc.,etc. Another aspect of the disclosure is used to improve the visibilitycharacteristics of our out-of-phase embedding techniques.

The foregoing and other aspects, features and advantages of the presentdisclosure will be even more apparent from the following detaileddescription, which proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 a is a diagram illustrating CMYK channels; and FIG. 1 billustrates the color CMYK channels of FIG. 1 a, embedded withinformation.

FIG. 2 illustrates hiding an image in media.

FIG. 3 is a flow diagram illustrating an embedding method according toone implementation of the present disclosure.

FIGS. 4 and 5 are graphs showing hidden signal strength in terms ofluminance.

FIGS. 6 and 7 are graphs showing hidden signal strength in terms ofcolor saturation.

FIG. 8 illustrates limiting a signal tweak in low CMY areas to reducehidden signal visibility.

FIG. 9 illustrates the segmentation of media into blocks.

FIG. 10 illustrates a feedback loop in an embedding process.

FIG. 11 illustrates feedback for the FIG. 10 feedback loop.

FIGS. 12 a and 12 b illustrate detection apparatus.

FIG. 13 illustrates orientation fiducials hidden in a media signal withour out-of-phase embedding techniques.

FIG. 14 illustrates out-of-phase embedding of a spot color.

FIG. 15 illustrates a printer calibration process.

FIG. 16 illustrates embedding in a blue channel with offsettingembedding occurring in red and green channels.

DETAILED DESCRIPTION Image Embedding

With reference to FIG. 2, an image 10 is steganographically hiddenwithin media 12. Of course, media 12 may represent digital media such asan image, photograph, video frame, graphic, picture, logo, product tag,product documentation, visa, business card, art work, brochure,document, product packaging, trading card, banknote, deed, poster, IDcard (including a driver's license, member card, identification card,security badge, passport, etc.), postage stamp, etc., etc. And image 10can correspond to a digital representation of a photograph, picture,graphic, text, orientation fiducial, object, barcode, message, digitalwatermark, outline, symbol, etc., etc. In the FIG. 2 example, image 10includes a close-up photograph, and the media includes a driver'slicense or passport photograph. The hiding (or embedding) isaccomplished using our inventive out-of-phase techniques.

With reference to FIG. 3, our K-phase hiding is preferably initiated byconverting image 10 to a black channel image 10′ (step 30—FIG. 3). Mostdigital imaging software tools such as Adobe's Photoshop facilitate sucha black channel conversion. The black channel image 10′ includes a setof black pixel values (e.g., gray-scale values) 10′. A location in themedia 12 is selected to place the black channel image (step 32). Thedashed circle 13 in FIG. 2 represents this selected location. The media12 location can be represented by sets of media 12 pixels. (For example,a first set of pixels corresponds to the selected location's blackchannel values, a second set corresponds to the selected location's cyanchannel values, a third set corresponds to the selected location'smagenta channel values, and a fourth set corresponds to the selectedlocation's yellow channel values). The set of black channel image 10′values is applied to the black channel pixels in the selected locationof media 12—effectively modifying media 12 (step 34). For example, if animage 10′ pixel includes a gray-scale value of 3, this gray-scale valueis applied to a corresponding pixel in the selected media 12 location toraise that corresponding pixel value by 3. In an alternativeimplementation, instead of adjusting the corresponding pixel in theselected media 12 location by the gray-scale value, we replace thatcorresponding pixel value with the black image 10′ gray-scale value. Inanother implementation, the corresponding media 12 pixel is modified toachieve the gray-scale value of the image 10′ pixel. Of course we canscale and/or weight the gray-scale value as needed prior to modifyingpixels in the selected location of media 12.

The black channel image 10′ is inverted to produce a set of signaltweaks (step 36). For example, if a black channel pixel is tweaked by agrayscale value of say 24, then a corresponding, inverted CMY tweakvalue is −24. (As an alternative implementation, image 10 is convertedinto corresponding C, M and Y images and such images are applied totheir respective channels. These signal tweaks are then used to modifyor change the color values in their respective CMY color channels (step38). Most preferably, in the above example, the −24-tweak value isapplied to each of the CMY color channels. The overall luminancecancellation can be effected as such. In another implementation weunevenly spread the tweak value over the CMY channels to achieve anoverall luminance change in a given media location to cancel the +24tweak in the black channel. For example, if using a luminance equationof: L=0.3*C+0.6*M+0.1*Y, we can achieve an overall luminance tweak of−24 by tweaking C=−15, M=−30 and Y=−15. Of course there is a vast rangeof other color combinations to achieve the same collective luminancechange. Care should be taken, however, to minimize a color shift whenusing this tweak-spreading alternative. The CMY pixels and the K pixelsare thus out-of-phase with respect to one another—resulting in a localcancellation of the perceived luminance change. Accordingly, image 10 issuccessfully hidden or carried by media 12.

The selected location 13 can be determined manually, e.g., via editingsoftware tools (like Adobe's Photoshop). Or the selection process can beautomated.

Image Hiding Enhancements

We have developed improvements to enhance our out-of-phase hidingtechniques. These improvements apply to hiding both images and digitalwatermark signals (in this section both will be referred to as a hiddensignal). While these techniques are not necessary to carry out ourout-of-phase hiding techniques, they generally reduce the visibility ofa hidden signal. Consider our following inventive improvements.

High Luminance Areas

Media 12 may include areas of low CMY and/or K ink (or signalintensity). In a first case, an area includes little or no C, M and/or Yink. This results in an inability to counteract (or cancel) an invertedsignal in a corresponding channel(s). Accordingly, we can sample theluminance of a media 12 area (or pixel) and, based on the luminancelevel, determine whether to scale back the hidden signal strength. Forexample, we begin to scale back the signal strength once the luminancereaches a predetermined threshold (e.g., in a range of 70-95%luminance). We can scale back the signal strength for a given areaaccording to a linear reduction, as shown in FIG. 4, or we can scale thesignal strength in a non-linear manner, e.g., as shown in FIG. 5. Theillustrated scaling signal strength applies to both the K channel andCMY channels. In a related implementation, we determine the luminance ofthe yellow channel. We base our scaling decisions on the yellowluminance percentage.

Saturated Color

Hiding signals in a saturated color area can also result in increasedhidden signal visibility concerns. For this document the term“saturation” refers to how pure a color is, or refers to a measure ofcolor intensity. For example, saturation can represent the degree ofcolor intensity associated with a color's perceptual difference from awhite, black or gray of equal lightness. We determine the colorsaturation level in a color plane (e.g., the yellow color plane), andthen scale back a hidden signal strength as the color saturation levelexceeds a predetermined level (e.g., 80% yellow color saturation). Aswith the FIGS. 4 and 5 implementations, we can scale the signal strengthin a linear manner (FIG. 6) or in a non-linear manner (FIG. 7).

Low or High Luminance Areas

We have found that we can even further improve the visibilitycharacteristics of our hidden signals by considering the amount ofluminance at a given pixel or other media 12 area. A low luminance mayindicate that there is insufficient CMY to compensate for a K channeltweak. For example, a 10% luminance in CMY for a given pixel impliesthat the pixel can accommodate only about a 10% signal tweak (e.g.,remember the simplified luminance relationship mentioned above:L=0.3*C+0.6*M+0.1*Y). With reference to FIG. 8, we can cap (or limit)the positive K tweak signal level in such low CMY areas to ensure thatthe CMY levels can be sufficiently decreased to counteract or cancel thepositive K channel signal.

Similarly, in an area of high CMY luminance, a negative K channel tweakcan be capped (or limited) to ensure a sufficient range to increase theCMY values.

Equalizing Detectability

Now consider an implementation where media 12 is segmented into aplurality of blocks (FIG. 9). Here a block size can range from a pixelto a group of pixels. We redundantly embed an image or watermark signalin each of (or a subset of) the blocks. As shown in FIG. 10, wepreferably use signal feedback (k) to regulate the embedding process. Asignal feedback (k) method is shown in FIG. 11. A black (K) channelimage or watermark signal (in this section hereafter both referred to asa “watermark”) is embedded in block i of media 12 (step 110), where “i”is an integer ranging from 1−n and where n is the total number ofblocks. The watermark signal is inverted (step 112) and embedded in theCMY channels of block i (step 114). At this point, we preferably performa detection process of the signal embedded within the i^(th) block (step116). The detection process determines whether the signal issufficiently detectable (step 118). The term “sufficient” in thiscontext can include a plurality of levels. In one, “sufficient” impliesthat the signal is detectable. In another, the detectability of thesignal is ranked (e.g., according to error correction needed, ease ofdetectability, or a detection-reliability metric, etc.). The termsufficient in a ranking context also implies that the detection rankingis above a predetermined threshold. The process moves to embed a newblock i+1 if the embedding is sufficient (120). Otherwise the signalstrength is increased or otherwise altered (step 122) and the embeddingof block i is repeated.

Such a signal feedback process helps to ensure consistent embeddingthroughout media 12.

Infrared Image Detection

An infrared detection method is illustrated with reference to FIG. 12 a.In particular, the illustrated detection method employs infraredillumination to facilitate image (or watermark) detection. Media 12 isilluminated with an infrared illumination source 14. The media 12 isembedded as discussed above, for example, to include various componentsin a multicolor dimension space (e.g., CMYK). A first component (orimage) is preferably embedded in the CMY channels. A second component(or image) is embedded in the K channel. The second component ispreferably inverted (or is out-of-phase) with respect to the CMYchannels.

Infrared illumination source 14 preferably includes a light emittingdiode, e.g., emitting approximately in a range of 800 nm-1400 nm, or aplurality of light emitting diodes (“LED”). Of course, there are manycommercially available infrared diodes, and such may be suitable usedwith our present detection techniques. It will be appreciated that manycommercially available incandescent light sources emit light both in thevisible and infrared (“IR”) spectrums. Such incandescent light sourcesmay alternatively be used as infrared illumination source 14. Indeed,infrared watermark detection may be possible in otherwise normal(“daylight”) lighting conditions, particularly when using an IR-passfilter.

A conventional power source powers the infrared illumination source. (Wenote that a variable trim resistor and a small wall transformer can beoptionally employed to control illumination source 14). Poweralternately can be supplied from a battery pack, voltage or currentsource, or by directly tapping a power source of a camera, e.g.,internally drawn from a parallel, USB, or corded power lines. For aconsumer device, a battery pack or a single power cord that is steppeddown inside a digital watermark reader housing can also be used.

Returning to the composition of an out-of-phase hidden image (orwatermark), a first image (or watermark) component is embedded in a K(or black) channel. A second image component, e.g., which isout-of-phase with respect to the K channel, is embedded in the CMYchannels. These characteristics have significance for infrareddetection. In particular, C, M and Y inks will typically have hightransmission characteristics in the infrared spectrum when printed,which render them nearly imperceptible under infrared illumination. Yetconventional black inks absorb a relatively high amount of infraredlight, rendering the black channel perceptible with infraredillumination. We note that standard processing inks, such as thoseconforming to the standard web offset press (SWOP) inks, include blackink with IR detection properties. Of course, there are many other inksthat may be suitably interchanged in the present disclosure.

As discussed above our out-of-phase embedding provides an effectivecancellation of perceived luminance changes when the CMYK image isviewed in the visible spectrum. Indeed, combining an inverted watermarksignal “tweak” or “bump” in a K channel with a correspondingnon-inverted watermark signal tweak or bump in the CMY channelseffectively cancels an overall perceived luminance change. However,under infrared illumination, the hidden image (or watermark) componentin the black (K) channel becomes perceptible without interference fromthe C, M and Y channels. An infrared image primarily portrays (e.g.,emphasizes) the black channel, while the C, M and Y channels areeffectively imperceptible under infrared illumination.

In one implementation, camera 16 captures an image of media 12.Preferably, camera 16 includes an IR-Pass filter that passes IR whilefiltering visible light. For example, the Hoya RM90 filter availablefrom M&K Optics L.L.C. is one of many IR-Pass/Visible Opaque filterssuitable for daylight detection. Another suitable filter is the RG850filter, part number NT54-664, available from Edmund Scientific. Thesefilters are offered as examples only, and certainly do not define theentire range of suitable IR-pass filters. Of course there are many otherIR-Pass filters that are suitably interchangeable with the presentdisclosure.

In yet another implementation, a conventional digital camera (or webcam) is modified so as to capture infrared light. In particular, mostdigital cameras and web cams include an IR filter, which filters out IRlight. Removing the IR filter allows the camera to capture light in theIR spectrum. Consider a visibly dark environment (e.g., an enclosedcase, shielded area, dark room, etc.). Media 12 is illuminated byinfrared illumination source 14 in the visibly dark environment. Camera16 (without an IR filter) effectively captures an infrared image (i.e.,the K channel image) corresponding to the illuminated media 12.

The captured image is communicated to computer 18. Preferably, computer18 includes executable software instructions stored in memory forexecution by a CPU or other processing unit. If media 12 includes adigital watermark, the software instructions preferably includeinstructions to detect and decode the embedded digital watermark.Otherwise, the instructions preferably include instructions to displaythe K-phase image. The software instructions can be stored in memory orelectronic memory circuits. Of course, computer 18 can be a handheldcomputer, a laptop, a general-purpose computer, a workstation, etc.Alternatively, computer 18 includes a hard-wired implementation, whichprecludes the need for software instructions.

With reference to FIG. 12 b, a detection housing 20 can be provided tohouse an infrared illumination source 14 and digital camera (both notshown in FIG. 12 b, since they are within the opaque housing 20). Thehousing 20 is preferably opaque to shield (or otherwise constructed tofilter) the camera and media 12 from visible light. The housing 20 hasan opening 20 a to receive the media 12. In a first case, opening 20 ais adapted to engulf media 12. This allows media 12 to be placed on asurface (e.g., table, imaging station, or counter) and the housingopening 20 a to be placed over media 12, effectively shielding media 12from visible light. In a second case, the opening 20 a receives media 12into (e.g., slides media through opening 20 a) and positions media 12within the opaque housing 20. In either implementation, the infraredillumination source 14 illuminates media 12, or the digital camera 12captures an image of the illuminated media (e.g., captures as image ofthe K-channel image). The digital camera 12 communicates with computingdevice 14, which detects and decodes a digital watermark embedded withmedia 12, if present, or otherwise displays the image.

In another illustrative embodiment, the above described infrareddetection technique is carried out in a visibly dark environment, suchas a dark room, shielded area, etc. An out-of-phase image (or digitalwatermark) is embedded in media. The media is illuminated with aninfrared illumination source, and a digital camera captures an image ofthe illuminated media.

In still another illustrative embodiment, the above described infrareddetection technique is carried out in a visibly lighted environment. Anout-of-phase image (or watermark) is embedded in media. The media isilluminated with an infrared illumination source, and a digital cameracaptures an image of the media. Preferably, the camera includes anIR-pass filter. The digital camera communicates with a computing device,which detects and decodes an out-of-phase image (or digital watermark)embedded in the media.

Infrared detection is an elegant solution to detect out-of-phase imagesor digital watermarks, since high transmission colors in the IR spectrumare effectively washed out, allowing detection of a low transmissioncolor channel. Specialized inks are not required to embed theout-of-phase digital watermark. Indeed most multicolor printer inkpacks, offset ink, process inks, dye diffusion thermal transfer inks,such as inks conforming to the SWOP standard include black inks thatallow infrared detection. Some of these inks include a carbon-basedblack ink, furthering the absorption of IR. While infrared detection isideal for out-of-phase images or digital watermarks, this method is alsoapplicable to detection of conventional digital watermarks. Forinstance, a watermark signal can be embedded only in a black channel ofmedia. Infrared illumination helps to reveal the embedded watermark inthis black channel. Alternatively, a digital watermark is embeddedacross many color planes, while detection is carried out in only thosecolor planes that are perceptible with IR illumination. Additionally,while we have discussed infrared detection techniques, we note thatultraviolet (UV) detection is also possible. In this case, one of thecolor channels (including the K channel) preferably includes UV pigmentsor properties. A UV detection process is carried out in a manneranalogous to that discussed above. (We also note that a CMY color caninclude IR/UV pigments or properties to facilitate detection of thatcolor with respective IR or UV detection methods).

Applications

Now consider a few applications of our inventive out-of-phase hidingtechniques.

Identification Documents (e.g., Passports, Driver's Licenses, etc.)

An out-of-phase image is hidden in an identification document to provideenhanced security. For example, a hidden image is a gray-scale versionof the identification document's photograph. An airport screener, or lawenforcement officer, illuminates the out-of-phase image with infrared(or ultraviolet) light for comparison of the hidden image to the printedphotograph. Or, instead of a photograph, the hidden image may includetext, which can be compared with the visibly printed text on theidentification document.

In assignee's U.S. Published Patent Application No. US 2002-0170966 A1,we disclosed various security and authentication improvements. Onedisclosed improvement ties machine-readable code such as barcodeinformation to a digital watermark. Our inventive out-of-phase hidingtechniques can be used with the techniques disclosed in theabove-mentioned application. For example, instead of hiding anout-of-phase image in the identification document, we instead embeddedan out-of-phase digital watermark. The digital watermark includes apayload, which has information corresponding to the printed informationor to information included in a barcode. In one implementation, theinformation includes a hash of the barcode information. In anotherimplementation, we hid a barcode in the identification document asdiscussed below.

Hiding Bar Codes in Out-of-Phase Channels

Over the years, a number of standards organizations and private entitieshave formed symbology standards for bar codes. Some examples ofstandards bodies include the Uniform Code Council (UCC), EuropeanArticle Numbering (EAN, also referred to as International ArticleNumbering Association), Japanese Article Numbering (JAN), HealthIndustry Bar Coding Counsel (HIBC), Automotive Industry Action Group(AIAG), Logistics Application of Automated Marking and Reading Symbols(LOGMARS), Automatic Identification Manufacturers (AIM), AmericanNational Standards Institute (ANSI), and International StandardsOrganization (ISO).

The UCC is responsible for the ubiquitous bar code standard called theUniversal Product Code (UPC). AIM manages standards for industrialapplications and publishes standards called Uniform Symbology Standards(USS). Some well know bar code schemes include UPC and UCC/EAN-128,Codabar developed by Pitney Bowes Corporation, 12 of 5 and Code 128developed by Computer Identics, Code 39 (or 3 of 9) developed byIntermec Corporation, and code 93.

Some bar codes, such as UPC, are fixed length, while others are variablelength. Some support only numbers, while others support alphanumericstrings (e.g., Code 39 supports full ASCII character set). Someincorporate error checking functionality.

While the bar codes listed above are generally one-dimensional in thatthey consist of a linear string of bars, bar codes may also betwo-dimensional. Two dimensional bar codes may be in a stacked form(e.g., a vertical stacking of one-dimensional codes), a matrix form, acircular form, or some other two-dimensional pattern. Some examples of2D barcodes include code 49, code 16 k, Data Matrix developed by RVSI,QR code, micro PDF-417 and PDF-417.

For more information on bar codes, see D. J. Collins, N. N. Whipple,Using Bar Code-Why It's Taking Over, (2d ed.) Data Capture Institute; R.C. Palmer, The Bar Code Book, (3^(rd) ed.) Helmers Publishing, Inc., andP. L. Grieco, M. W. Gozzo, C. J. Long, Behind Bars, Bar CodingPrinciples and Applications, PT Publications Inc., which are hereinincorporated by reference.

A hidden, out-of-phase image can include a barcode. Consider the vastpossibilities. A barcode is often disdained for aesthetic reasons, but ahidden, out-of-phase barcode can carry relatively large amounts ofinformation while remaining virtually imperceptible. In oneimplementation, a barcode is redundantly hidden or titled throughoutmedia using our out-of-phase embedding techniques. This allows forrobust barcode detection even if only a portion of the media isrecoverable. In another implementation one or more barcodes are placedin predetermined areas throughout the image. In still anotherimplementation, a barcode reader, such as those provided by Symbol(e.g., the VS4000 and P300IMG models) or Welch Allyn (e.g., the Dolphinmodel), is augmented with an infrared illumination source and/orIR-filters. Once illuminated, the barcode reader detects and decodes abarcode hidden in a K channel.

Fiducials and Orientation Signal

In some digital watermarking techniques, the components of the digitalwatermark structure may perform the same or different functions. Forexample, one component may carry a message, while another component mayserve to identify the location or orientation of the watermark in asignal. This orientation component is helpful in resolving signaldistortion issues such as rotation, scale and translation. (Furtherreference to orientation signals can be made, e.g., to previouslymentioned application Ser. No. 09/503,881). In some cases, channelcapacity is congested by an orientation signal.

One improvement is to embed an orientation signal using our out-of-phasehiding techniques. The message component of a digital watermark can thenbe embedded using out-of-phase or non-out-of-phase embedding techniques.This improvement will increase message capacity, while improvingvisibility considerations. Scale, orientation, and image translation canbe resolved based on the orientation of the fiducial.

A related improvement embeds a plurality of fiducials or orientationmarkers 54 in an out-of-phase channel of media 12 (FIG. 13). A watermarkdetection module detects the fiducials to identify distortion.

Spot Colors

We have found that our inventive techniques are not limited to processcolors. Indeed, our out-of-phase techniques can be extended to spotcolors. (See Assignee's U.S. patent application Ser. No. 10/074,677,filed Feb. 11, 2002 (now U.S. Pat. No. 6,763,124), for a furtherdiscussion of spot colors and digitally watermarking spot colors. TheU.S. Pat. No. 6,763,124 patent is hereby incorporated by reference).With reference to FIG. 14, and preferably (but not limited to)relatively darker spot colors, e.g., violets, blues, etc., we counteracta watermark signal (or image) embedded in the spot color channel with aninverted signal in a K channel. Preferably, the K channel base intensityis subtle (e.g., 0% as represented by the K channel base level dashedline in FIG. 14) in comparison to the base level spot color intensity(e.g., 100% intensity as represented by the spot color maximum leveldashed line in FIG. 14). The watermark signal (or image) signal isembedded through a combination of negative spot color tweaks andpositive, offsetting, K channel tweaks. Infrared illuminationfacilitates detection of the K-channel watermark tweaks. (Embedding aspot color need not be limited to negative tweaks. Indeed, if the spotcolor is not at 100% intensity, positive spot color tweaks andcorresponding negative K channel tweaks can facilitate embedding).

Paper Information and Printing Processes

Another improvement is to carry printing process information and/orpaper characteristics with a digital watermark. For example, a digitalwatermark may include signal gain or embedding characteristics that arespecific to a printing press, printing process, process ink type orpaper characteristics. The digital watermark can be embedded in adigital file, which is analyzed prior to a print run. The embeddingprocess is adjusted according to the watermark data. Or the watermarksignal can be analyzed after printing one or more test copies. Thesignal strength or payload metric can be analyzed to determine whetherthe process should be adjusted.

Our out-of-phase digital watermark can be used to detect a misalignmentin a printing process. With reference to FIG. 15 a printer 150 outputs aCMYK (or spot color, etc.) printed sheet 152. The printed sheet includesan out-of-phase digital watermark or image hidden therein. An inputdevice 154 captures an image of sheet 152. Preferably, input device 154captures a visible spectrum image of sheet 152. The input deviceprovides the captured image (e.g., digital scan data) to a watermarkdetector 156. The watermark detector 156 analyzes the captured image insearch of the embedded out-of-phase digital watermark. The watermarkdetector 156 should not be able to detect the embedded watermark if theprinting of the CMY and K are aligned, due the localized cancellation ofthe signal tweaks (or luminance changes). The term aligned in thiscontext implies that the CMY and K are sufficiently inverted to allowlocalized cancellation. A misalignment is identified if the watermarkdetector 156 reads the digital watermark. Such a misalignment isoptionally communicated from the watermark detector 156 to the printer150 or otherwise provided to announce the printing misalignment. Ofcourse other alignment and color balance information can be identifiedfrom the detection of the digital watermark.

Color Channel Keys

A related inventive technique embeds a key in one color channel fordecoding a watermark in a second color channel. Consider animplementation where a first digital watermark is embedded in a firstcolor channel. The first digital watermark includes a payload includinga key. The key is used to decode a digital watermark embedded in asecond color plane. The term decode in this context includes providing areference point to locate the second watermark, providing a key tounlock, decrypt, decode or unscramble the second digital watermarkpayload, etc. Of course this inventive technique is not limited to ourout-of-phase digital watermarks.

Fragile Security

Our out-of-phase hiding techniques are fragile since a signal processingoperation that combines the K channel with the CMY channels effectivelycancels the hidden signal. A fragile watermark is one that is lost ordegrades predictably with signal processing. Conversion to other colorspaces similarly degrades the watermark signal. Take a typicalscan/print process for example. Digital scanners typically have RGBimage sensors to measure the image color. Scanning an out-of-phaseembedded CMYK image degrades the embedded watermark due to thecombination of K with CMY in a local area, effectively canceling thewatermark. When the RGB image representation is converted to CMYK andprinted, the watermark signal is effectively lost. Similarly, otherconversions, such as to an L*a*b color space, degrade the out-of-phasewatermark due to the combination of K with CMY throughout local areas.Nevertheless, the watermark signal is detectable from an original CMYKmedia, since the K channel can be detected separately by viewing, e.g.,in the near infrared.

A fragile watermark has utility in many applications. Takecounterfeiting, for example. The inventive fragile watermark is embeddedin original CMYK media. If the media is copied, the embedded fragilewatermark is either lost or degrades predictably. The copy is recognizedas a copy (or counterfeit) by the absence or degradation of the fragilewatermark. Fragile watermarks can also be used in conjunction with otherwatermarks, such as robust watermarks. The fragile watermark announces acopy or counterfeit by its absence or degradation, while the otherrobust watermark identifies author, source, links and/or conveysmetadata or other information, etc. In other embodiments, a fragilewatermark is an enabler. For example, some fragile watermarks mayinclude plural-bit data that is used to enable a machine, allow accessto a secure computer area, verify authenticity, and/or link toinformation. This plural-bit data is lost or sufficiently degrades in acopy, preventing the enabling functions.

Another inventive feature is to embed a hash or other representation ofa product (e.g., product code or serial number) in a digital watermarkpayload or message. The digital watermark is then tied or linkeddirectly to the product. If the product includes a barcode having theproduct code, such can be compared with the digital watermark.

Imperceptible Embedding

Our inventive techniques provide a very imperceptible digital watermark,particularly for printed images. One advantage of our embeddingtechniques is that a relatively strong signal can be inserted whilestill minimizing visibility to the human eye. In one implementation wetake advantage of low sensitivity of the human visual system to highfrequency blue/yellow (e.g., chrominance). With reference to FIG. 16 ablue signal tweak (e.g., representing a watermark component in terms ofpixel color values) is calculated. The signal tweak can be in the formof a spatial image change (e.g., pixel color values) or frequency domainchange. (If a frequency change, the change is preferably converted to aspatial domain adjustment so that an offsetting signal change can bedetermined). In fact a watermarking signal, e.g., as described inassignee's U.S. Pat. Nos. 6,614,914 and 6,122,403, can be provided andsuch a signal can be used as a blue channel tweak. (In actuality therewill by many such tweaks spread over an image in various locations.) Aninverted or offsetting signal tweak is then determined for the red andgreen channels at a corresponding image area (e.g., correspondingspatial or pixel location, but in the different channels). One goal ofthe inverted signal is to provide a resulting image with constantluminance at the various embedding areas. For each tweak in the bluechange, we preferably provide an offsetting tweak in the red and/orgreen channels. This offsetting tweak cancels or offsets localizedluminance changes attributable to the blue channel change. We have foundthat an inverted or offsetting signal tweak of minus ⅛ of the blue tweakthat is applied to each of the red and green color channels helpsmaintain constant luminance in image or video areas receiving signaltweaks. (For example is a signal tweak of 16 is applied to a blue pixelor group of pixels, a minus 2 signal tweak is applied to a correspondingpixel or group of pixels in the red and green channels.) Thus, thewatermark signal is effectively conveyed in chrominance. (While weprefer a ⅛ tweak change in each of the red and blue channels, someluminance cancellation is found as the minus tweak values range fromabout 1/16 to ¼.) We sometimes—affectionately—refer to this type ofdigital watermark embedding as “blue phase” embedding.

The “tweaked” or embedded color channels are provided to a printer forprinting. We note that most of today's printers and/or printer drivershave sophisticated color converters that convert RGB signals into CMY orCMYK signals for printing. Those of ordinary skill in the art will knowof different color converting techniques as well. Our above blue phasewatermarking survives this color conversion quite robustly.

Watermark detection of a printed document includes presenting theprinted image to an optical scanner. The optical scanner captures scandata corresponding to the printed image, preferably including scan datarepresenting (or converted to) red, green and blue channels. We cancombine the color channels to help emphasize the watermark signal andminimize image interference. For example, we preferably scale andprocess the color channels per pixel color or chrominance values asfollows:Detection Signal (chrominance)=0.5*blue−0.25*(red+green)+128.The scaling of color channels is chosen to minimize image interference(e.g., color channels are subtracted) and avoid saturation, e.g., ifcolor data is being represented as an 8 bit value. The 128 pixel coloror grayscale value helps shift a color value to avoid color saturation.Of course this shifting value can range depending on imagecharacteristics, detector requirements, etc. For example, the shift canbe in a color value (e.g., often represented as a grayscale value for aparticular color channel) range of about 64-192. Acceptable detectionmay also occur when the blue channel is scaled in a range of 0.3-0.75and the red+green are scaled proportionally in a range of 0.15-0.375.

Since the watermark signal is effectively conveyed in the chrominancechannel, we have found that this type of watermarking is somewhatsusceptible to JPEG compression. Nevertheless, while print applicationsare one of the main areas of application for these blue phasetechniques, there are many other areas that will benefit from thesetechniques as well, e.g., digital cinema. Our blue phase techniques areused to embed a digital watermark signal in a video signal after it isdecompressed, but before (or as) it is being projected on the screen.That is, the uncompressed data stream is feed into a digital watermarkembedder. The various color channels are embedded as discussed above.The projected video includes a blue phase watermark. The watermark caninclude a plural-bit payload that, e.g., identifies the projector,theater, date/time, movie, etc. We can add a buffering system to ensurethat the perceived video—from the paying customer's point of view—isuninterrupted.

Another application is a combination of a blue phase watermark withother types of watermarks (e.g., luminance based watermark). Chrominanceand luminance are generally orthogonal. This allows for little or nointerference between these types of watermarks. Different watermarkcomponents can be conveyed with each type of watermark. For example, achrominance based watermark can include a so-called watermarkorientation component while a luminance based watermark includes amessage or payload that is synchronized according to the watermarkorientation component. The message or payload can vary across an image(e.g., the plural-bits of the message change according to spatiallocation) while the orientation component remains constant. This isparticularly helpful in map or geo-location applications, wheredifferent image regions represent different geo-locations. The messagesor payloads can represent or link to geo-location information. Thecurious reader is directed to the following related applications: US2002-0122564 A1; US 2002-0124171 A1; US 2002-0135600 A1 and US2004-0008866 A1, which are each hereby incorporated by reference. Ifusing two types of watermarking, a detector can be constructed thatanalyzes different frames under different detection protocols. Forexample, a first frame is analyzed according to the blue phase detectionmentioned above. A second frame is analyzed to detect a luminance (orother) based watermark. A third frame is again analyzed to detect a bluephase watermark, etc.

We have also found that our blue phase watermarking provides strongdetection results in many of today's handheld readers (e.g., cellphones, PDA, etc).

Of course our blue phase embedding techniques can be used with the manyother implementations and features discussed in this and theincorporated by reference patent documents. For example, instead ofembedding a watermark signal, we can embed an image or 2D barcode withblue phase techniques. For every blue phase change to represent an imageor 2D barcode, we can introduce a corresponding and offsetting change inred and green—in hopes of maintain constant luminance in embeddingareas.

CONCLUSION

Preferably, an out-of phase watermark signal is embedded 180 degreesout-of-phase with corresponding channels. However, some cancellationwill still be achieved if the signal is approximately 180 degrees, forexample, in a range of ±0-50% from the 180-degree mark. The term“inverted” includes values within this range. We note that while thepresent disclosure has been described with respect to CMYK process inks,the present invention is not so limited. Indeed, our inventivetechniques can be applied to printing processes using more than fourinks with the K channel canceling the three or more color channels.Similarly, as shown above under the spot color discussion, our inventivetechniques are also applicable to printing processes using less thanfour inks. Of course our techniques can be used with a variety ofprinting techniques, including offset printing, dye diffusion thermaltransfer (D2T2), other thermal transfers, process ink printing, etc.,etc., etc.

The section headings in this application are provided merely for thereader's convenience, and provide no substantive limitations. Of course,the disclosure under one section heading may be readily combined withthe disclosure under another section heading.

To provide a comprehensive disclosure without unduly lengthening thisspecification, the above-mentioned patents and patent applications arehereby incorporated by reference, along with U.S. Pat. No. 6,763,122.The particular combinations of elements and features in theabove-detailed embodiments are exemplary only; the interchanging andsubstitution of these teachings with other teachings in this applicationand the incorporated-by-reference patents/applications are alsocontemplated.

The above-described methods and functionality can be facilitated withcomputer executable software stored on computer readable media, such aselectronic memory circuits, RAM, ROM, magnetic media, optical media,memory sticks, hard disks, removable media, etc., etc. Such software maybe stored and executed on a general purpose computer, or on a server fordistributed use. Data structures representing the various luminancevalues, out-of-phase embedded signals, embedded color planes, colorsignals, data signals, luminance signals, etc., may also be stored onsuch computer readable media. Also, instead of software, a hardwareimplementation, or a software-hardware implementation can be used.

In view of the wide variety of embodiments to which the principles andfeatures discussed above can be applied, it should be apparent that thedetailed embodiments are illustrative only and should not be taken aslimiting the scope of the invention. Rather, we claim as our inventionall such modifications as may come within the scope and spirit of thefollowing claims and equivalents thereof.

1. A method comprising: receiving optically captured data associatedwith printed media, wherein the received captured data represents a bluechannel, a green channel and a red channel, and wherein a first digitalwatermarking is embedded in the printed media; scaling the blue channelwith a first factor; scaling a sum of the red channel and the greenchannel with a second factor; subtracting the scaled sum of the redchannel and the green channel from the scaled blue channel and adding ashift factor to yield a detection signal; and searching, using aprocessor, for the first digital watermarking in the detection signal.2. The method of claim 1, wherein scaling the blue channel, scaling thesum of the red channel and the green channel, subtracting and adding arerelated according to:first factor*blue channel−second factor*(red channel+greenchannel)+shift.
 3. The method of claim 1, wherein the first factor is ina range of 0.3-0.75, the second factor is in a range of 0.15-0.375, andthe shift factor is in a color value range of 64-192.
 4. The method ofclaim 1, wherein the first factor comprises 0.5, the second factorcomprises 0.25 and the shift factor comprises
 128. 5. The method ofclaim 1, wherein the detection signal represents chrominancecharacteristics.
 6. The method of claim 1, further comprising searchingfor a second digital watermarking from luminance characteristicsassociated with the blue channel, the green channel and the red channel.7. An apparatus comprising: a processor configured to: receive opticallycaptured data associated with printed media, wherein the receivedcaptured data represents a blue channel, a green channel and a redchannel, and wherein a first digital watermarking is embedded in theprinted media; scale the blue channel with a first factor; scale a sumof the red channel and the green channel with a second factor; subtractthe scaled sum of the red channel and the green channel from the scaledblue channel and adding a shift factor to yield a detection signal; andsearch for the first digital watermarking in the detection signal. 8.The apparatus of claim 7, wherein the processor is configured tocalculate the detection signal according to:first factor*blue channel−second factor*(red channel+greenchannel)+shift.
 9. The apparatus of claim 7, wherein the first factor isin a range of 0.3-0.75, the second factor is in a range of 0.15-0.375,and the shift factor is in a color value range of 64-192.
 10. Theapparatus of claim 7, wherein the first factor comprises 0.5, the secondfactor comprises 0.25 and the shift factor comprises
 128. 11. Theapparatus of claim 7, wherein the detection signal representschrominance characteristics.
 12. The apparatus of claim 7, wherein theprocessor is further configured to search for a second digitalwatermarking from luminance characteristics associated with the bluechannel, the green channel and the red channel.
 13. A non-transitorytangible computer-readable medium having instructions stored thereon,the instructions comprising: instructions to receive optically captureddata associated with printed media, wherein the received captured datarepresents a blue channel, a green channel and a red channel, andwherein a first digital watermarking is embedded in the printed media;instructions to scale the blue channel with a first factor; instructionsto scale a sum of the red channel and the green channel with a secondfactor; instructions to subtract the scaled sum of the red channel andthe green channel from the scaled blue channel and adding a shift factorto yield a detection signal; and instructions to search for the firstdigital watermarking in the detection signal.
 14. The tangiblecomputer-readable medium of claim 13, further comprising instructions tocalculate the detection signal according to:first factor*blue channel−second factor*(red channel+greenchannel)+shift.
 15. The tangible computer-readable medium of claim 13,wherein the first factor is in a range of 0.3-0.75, the second factor isin a range of 0.15-0.375, and the shift factor is in a color value rangeof 64-192.
 16. The tangible computer-readable medium of claim 13,wherein the first factor comprises 0.5, the second factor comprises 0.25and the shift factor comprises
 128. 17. The tangible computer-readablemedium of claim 13, wherein the detection signal represents chrominancecharacteristics.
 18. The tangible computer-readable medium of claim 13,further comprising instructions to search for a second digitalwatermarking from luminance characteristics associated with the bluechannel, the green channel and the red channel.